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Abstract. This article introduces a formal model to specify, model and 
validate hierarchical complex systems described at different levels of anal- 
ysis. It relies on concepts that have been developed in the multi-agent- 
based simulation (MABS) literature: level, influence and reaction. One 
application of such model is the specification of hierarchical complex 
systems, in which decisional capacities are dynamically adapted at each 
level with respect to the emergences/constraints paradigm. In the con- 
clusion, we discuss the main perspective of this work: the definition of a 
generic meta-model for holonic multi-agent systems (HMAS). 

Keywords: multi-level multi-agent based simulations, formal models, 
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1 Introduction 

Engineering a complex system such as a flexible manufacturing system (FMS) is a 
challenging problem. The target system is complex, holonic, relies on distributed 
decisional processes, and must be adaptive, i.e., robust to perturbations and 
easily reconfigurable. 

To solve these problems, proposed solution^ take advantage of system 

— complexity, distributing the control in system components that embody 
primitive cognitive capacities, e.g., be able to be identified, to communicate, 
to react to environmental changes, 

— holonic structure, using dedicated meta-models and conception methodolo- 
gies. 

E.g., heterarchical [3] or semi-heterarchical [25] control, holonic multi-agent systems 
(HMAS) 2 6 35 34 or intelligent product based concepts [24] . 



An important tool in the design, simulation and validation of such solutions 
has been multi-agent-based simulation (MABS). This article introduces a formal 
model to specify, model and and validate hierarchical complex systems. It takes 
inspiration from two trends in MABS research: 

— the formalization of interaction models, 

— multi-level modeling, where interacting agents are ontologically distributed 
among multiple layers of organization. 

The article is organized as follows: 

— in the section [2] the two trends of MABS research cited above, multi-level 
modeling and formal modeling, are introduced, 

— the section [3] presents a generic formal model for multi-level MABS, 

— an abstract implementation of this model, focusing on the specification of 
hierarchical multi-agent systems (MAS), in which decisional capacities are 
dynamically adapted at each level with respect to the emergences/constraints 
paradigm, is proposed in the section 21 

— the conclusion (section [5]) summarizes our contributions and perspectives. 

2 Two trends in MABS research 
2.1 Multi-level modeling 

A level represents a point of view on the system, and its relations to other 
points of view |16| . While this concept seems important to understand complex 
system^], it generally remains abstract: implementations tend to constraint this 
definition, in particular the relations between levels. Therefore, a multi-level 
model integrates knowledge on different levels and their relations. Multi-scale are 
multi-level models characterized by hierarchical relations in levels [7112119 23J. 
A level may represent, according to the context, a spatio-temporal extent, a 
position in a decision hierarchy, etc. Let consider these two examples. 

1 . The system is characterized by processes that have different spatio-temporal 
extents. Two types of relations can be commonly found in such models: 

— scaling, i.e., computing macroscopic (resp. microscopic) variables from 
microscopic (resp. macroscopic) processes, 

— grouping and degrouping (or aggregation and disaggregation) [6I20I26] . 
i.e., defining a process at a level as a group (resp. part) of processes 
(resp. a process) at an other level. 

2. Levels are characterized by decisional capacities; relations represent the emer- 
gence of new capacities and the constraint over existing capacities |15I17| . 

A level is often viewed as a level of organization. This concept is closely 
related to the notion of holon [6] . This aspect is discussed in the section [5l 

2 The presentation focuses on the influences — > reaction model (IRM). Other ap- 
proaches such as IODA |10| or based on DEVS |18| are not described. 

3 Multi- level approaches have proven useful in many domain such as statistics [8], 
chemistry [9111] . physics [25], hydrology [25] or biology |33j . 



2.2 The influences — > reaction model 



The influences — > reaction model (IRM) has been developed to address issues 
raised by the classical vision of action in Artificial Intelligence as the transfor- 
mation of a global state [5]: 

— simultaneous actions cannot be easily handled, 

— the result of an action depends on the agent that performs it but not on 
other actions, 

— the autonomy of agents is not respected. 

Basically, it decomposes action in two phases: agents and environment (mi- 
cro level) produce a set of influences, then the system (at macro level) reacts 
to influences; e.g., detects and solves influence conflicts such as in the platform 
Jaal0. As [T3] notes, "the influences [produced by an agent] do not directly 
change the environment, but rather represent the desire of an agent to see it 
changed in some way". Thus, reaction computes the consequences of agent de- 
sires and environment dynamics. In recent years, variants of IRM have been 
developed to handle specific situations |I3II6I36I37| . This presentation focuses 
on the influence reaction model for simulation (IRM4S) |13| . 

Let 8{t) £ A be the dynamic state of the system at time t: 

S(t)=<a(t), 1 (t)>, (I) 

where a(t) £ S is the set of environmental properties and j(t) G r the set 
of influences, representing system dynamics. The state of an agent a € A is 
characterized by its physical state (j> a £ <£ a with <P a £ £ (e.g.. its position) and 
its internal state s a £ S a (e.g., its beliefs). 

The evolution of the system from t to t + dt is a two-step process: 

1. agents and environment produce a set of influence^ j'(t) G F', 

2. the reaction to influences produces the new dynamic state of the system. 

An agent a £ A produces influences through a function Behavior a : A M> J". 
This function is decomposed into three functions executed sequentially: 

Pa(t) = Perception a (5(t)), (2) 

s a (t + dt) = Memorization a (Pa(t), s a (t)), (3) 
j' a (t) = Decision a (s a (t + dt)). (4) 
The environment produces influences through a function Natural^, : A i— > r': 

j' u (t) = NaturaL(6(t)). (5) 



4 http: //www . j anus-project . org/ Jaak 

5 the sets of producible influence sets and influences produced at t are denoted re- 
spectively r' and Y{t) to point out that the latter is temporary and will be used to 
compute the dynamic state of the system at t + dt. 



Then the set of influences produced in the system at t is: 



(6) 



After influences have been produced, the new dynamic state of the system is 
computed by a function Reaction : S x i~" M> Z\ such as: 



3 A generic meta-model for multi-level MABS 

In this section, a generic meta-model for multi-level MABS, called IRM4MLS, 
is presented^!. This model has the following interesting properties: 

— any valid instance can be simulated |27) . 

— simulation scheduling is logically distributed by level, 

— complexity of simulation algorithm can be optimized according to model 
structure. 

3.1 Specification of the levels and their interactions 

A multi-level model is defined by a set of levels L and a specification of the 
relations between levelfl Two kinds of relations are specified in IRM4MLS: 
an influence relation (agents in a level I are able to produce influences in a 
level I' ^ V) and a perception relation (agents in a level I are able to perceive 
the dynamic state of a level I' ^ I), represented by directed graphs denoted 
respectively < L,Ej > and < L, Ep >, where Er and Ep are two sets of edges, 
i.e., ordered pairs of elements of L. Influence and perception relations in a level 
are systematic and thus not specified in Ej and Ep (cf. eq. [5] and . 

E.g. ,V2,2' G L 2 , if Ep = {11'} then the agents of / are able to perceive the 
dynamic states of / and I' while the agents of I' are able to perceive the dynamic 
state of 

The in and out neighborhood in < L,Ei > (respectively < L, Ep >) are 
denoted Nf and Nj~ (resp. Np and Np) and are defined as follows: 



VI G L,Nj(l) (resp. Np(l)) = {1} U {/' e L : I'l G Ej (resp. E P )}, (8) 
VZ G L,Nf(l) (resp. Np(l)) = {1} U {V G L : W e E T (resp. E P )}, (9) 



E.g., Ml, I 1 G i 2 if Z' G Nj(l) then the environment and the agents of I are 
able to produce influences in the level I'; conversely we have I G Nf(l'), i.e., V 
is influenced by I. 

6 The dynamic aspects of the meta-model, i.e., simulation algorithms, are not de- 
scribed here. An exhaustive presentation can be found in [16] . 

7 The notion of level is here similar to the notion of brute space in the MASQ meta- 
model [25]. 



S(t + dt) = Reaction{a{t),^{t)). 



(7) 
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Fig. 1. Main concepts of IRM4MLS (cardinalities are specified in the UML fashion) 



3.2 Agent population and environments 

The set of agents in the system at time t is denoted A(t). VI e L, the set of 
agents belonging to I at t is denoted Ai(t) C A(t). An agent belongs to a level 
iff a subset of its physical state </> a belongs to the state of the level: 

Va e A(t),Vl eL,ae Ai(t) iff 3<t> l a (t) c o (t)|^(t) c a\t). (10) 

Thus, an agent belongs to zero, one, or more levels. As notes |29[ p. 815], the 
physical state of an agent in a level, i.e., its body, is "the manifestation of an 
agent in the environment and allows others to perceive it." An environment can 
also belong to multiple levels (cf. fig. U]). 

3.3 Action modeling 

The dynamic state of a level I S L at time t, denoted 5 l (t) 6 A 1 , is a tuple 
< it (t),7'(t) >, where <r l (t) £ U l and j l (t) £ r l are the sets of environmental 
properties and influences of I. 

The influence production step takes into account the influence and perception 
relations between levels: 

Va e Ai, Behavior l a : J] A lp i-4 JJ ph '■ ( n ) 

lpEN+(l) h£N+(l) 

Once influences have been produced, interactions between levels do not mat- 
ter anymore. Thus, the reaction function defined in IRM4S can be re-used: 

Reaction 1 : S L x T u ^ A 1 , (12) 

where Reaction 1 is the reaction function proper to each level. 

4 Engineering hierarchical complex systems with 
IRM4MLS 

4.1 The emergence/constraint paradigm 

In many MABS, processes are considered on the following 2-level relative hier- 
archy: 



micro 



macro. 



Arrows represent causality relations between levels. Dashing suggests that 
they are generally not explicitly defined but emerge from interactions between 
entities. A contrario, a multi-level approach considers these relations explicitly. 
In engineering applications, a level may rather represents a position in a decision 
hierarchy (cf. section l2Tj) . Two kinds of relation may be distinguished in such 
systems: emergence of new capacities and constraint over existing capacities [14J. 
Let consider an example in the domain of FMS engineering. In a case study on 
automated guided vehicle (AGV) control presented in [17] (cf. section l4~4l) . the 
model relies on the following relations: 

emergences 

I 1 
AGV deadlock solving. 

T . I 

constraints 

Macro agents (representing a set of "trapped" AGVs) emerge from micro agent 
interactions when an interaction pattern defined as a deadlock is detected, and 
then constraint their behaviors to solve it. While the notions of emergence and 
constraint were informally defined in |17| . formal definitions in the context of 
IRM4MLS are given in the following. 

4.2 IRM4MLS implementation 

Let L be a hierarchy and {/i, M} C L two hierarchically coupled levels, fi refer- 
ring to the micro level and M to the macro level. Thus, (respectively Am) de- 
notes the agents of the micro-level (resp. macro-level). The emergence/constraint 
paradigm supposes that Ei D {/j,M,Mfi}. 

WeL, 7 "(t) = {V(t),7* / ,7^, (J 7a M 'W, (J 7^'(*)}- (13) 

An emergence e at the level M is an influence that has the following proper- 
ties: 

— e belongs to the macro-level but not to the micro-level: 

e G r M but e i r», (14) 

— e cannot be produced by the behavior of an agent or the environment of M: 

Vt,e(£ (J Behavior™ (S(t))U Natural^ 1 (5(t)), (15) 

a£A M 



with 6(t)=< 5 M (t),S»(t) >. 



Emergent influences generally determine the life-cycle (creation, evolution, de- 
struction) of agents at the macro-level. 

A constraint over an influence i, denoted -i£, is the special kind of influence 
that has the following properties: 

— {i, belongs to the micro-level but not to the macro-level: 

{*, -*} c r" but {i, -*} £ r M , (i6) 

— -4 cannot be produced by the behavior of an agent or the environment of /j,: 

Vt,-.i£ (J Behavior%(5(t)) U Natural%(6(t)), (17) 

with S(t) =< 6 M (t),8»(t) >, 

— -4 inhibits i: 

if {i,-i}C 7 /"(t) then 
Reaction^ (a* (i), 7^'(t)) = Reaction^ \a^{t), l^'{t)\{i}). K ' 

4.3 Conception of hierarchical systems 

The approach described below can be viewed as a semi-heterarchical control one 
and takes advantage of complexity and hierarchical (not yet holarchical) orga- 
nization of the system, distributing the control by level. Heterarchical control 
methods rely on self-organization principle^ and therefore assume that the sys- 
tem is able to achieve its goals and is easily reconfigurable, i.e., that the normal 
functioning mode emerges from the interactions between system components 
(products, machines, simulated entities, etc.) that embody limited cognitive ca- 
pabilities (cf. introduction). However, the trajectory of such systems may lead 
to non desired attractors. 

The proposed methodology is presented in the fig. [2j The system is designed 
iteratively in a two-step process. 

1 . From an initial specification of the system, a model of the system in normal 
functioning mode, is defined and verified, i.e., that system components have 
the necessary cognitive capacities to perform their tasks. 

2. From non desired attractors exhibited by the simulation of the model, the 
control strategy may be designed and validated. However, it is likely that 
the specification of the system has be modified to do so, e.g., because a new 
decisional level is needed. 

The notion of influence is very general and therefore, may have many possible 
meanings. In this case, let 

8 Self-organized systems are generally characterized by the use of environment as a 
communication medium to carry local informations as well as positive and negative 
feedbacks. 
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Fig. 2. Engineering methodology 



— 7 ( '(£) be the capacities of each agent of a level I at time t, i.e., the tasks they 
can perform at the moment, 

— 7 ( (i) the actual affectation of tasks to agents; the only cognitive capacity 
required for agents is to expose services they may provide. 

Thus, Reaction 1 is a task assignment algorithm that computes < a l (t),^ l (t) > 
from < a l (t), J l '(t) >. Note that the hierarchical nature of the system allows to 
decompose the specification of the system S by level: 



i.e., task assignments for all functioning modes. 

That design should lead to the definition of reaction functions that control 
goal affectations. If such a function cannot be defined, then the system design 
is not valid and must be redefined. This process is iterated until a solution is 
found (cf. fig.©. 

4.4 Case study: AGV deadlocks in gradient field-based FMS 

The main functionalities of an intelligent transportation system (ITS) are: (1) 
transport assignment, (2) routing, (3) gathering traffic information, (4) collision 
avoidance, (5) deadlock avoidance |38| . 

Gradient field-based approaches, where AGV trajectories are computed from 
gradient fields, allow to implement efficient ITS in FMS [31 32J. A dedicated task 
assignment algorithm is generally used to ensure functionality 1, while function- 
alities 2-4 rely on AGV and shop self-organization properties. Thus, an AGV 
has two cognitives capabilities: sense attractive or repulsive force fields and emit 
a repulsive force field. Similarly, a shop is able to emit attractive fields to require 
products to process and give back the result to the system. A known problem 
of gradient field-based approaches is that a group of AGVs may be trapped 
in local minima that lead to a system deadlock [30 32 39J. However, this issue 
can be easily addressed by hierarchical control methods that compute explicit 
trajectories^. 



S = WiS 1 ) :VZ e L,W € A 1 }, 



(19) 



Readers interested in general, i.e., not gradient-field based approaches, deadlock 
avoidance techniques in FMS may refer e.g., to [114140] , 
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AGV *■ > Shop AGV «■ > Shop 

(a) Initial design (b) Final design 

<- ■> mediated interactions 

Fig. 3. Decisional levels in the case study on AGV control 



The first design of the system is presented in fig. 3(a) a task assignment 
algorithm affects goals to AGVs (statically, a signal to maximize) and shops 
(dinamically, products to process). The deadlock avoidance functionality is not 
explicitly programmed but is supposed to emerge from mediated interactions 
between AGVs and shops. Various researches have shown that such a solution 
may reduce the number of deadlock occurrences but not eliminate it: routing 
is not deadlock avoidance [39|. A new system architecture is then designed (cf. 
fig. 3(b) |: if a deadlock (reified by an emergence) is observed by a deadlock 



solving alg orithm, constraints over signal sensing and emission are computed to 
solve it 



5 Conclusion 

In this article, we have presented a formal model for MABS and its implementa- 
tion to engineer hierarchical complex systems. Two types of influences have been 
distinguished in this approach : emergence, that basically triggers a new system 
behavior when a specific pattern is detected (in the previous short example of 
gradient field-based FMS, the detection of a deadlock triggers the modification 
of AGV repulsive signal emission) and constraint, that, as its name suggests, 
constraints decisional capacities of system entities to solve a situation. 

The main advantage of this approach lies in the multi-level and simulation 
capabilities of IRM4MLS, to model a system in which decisional capacities are 
distributed in its components and evolve along time to meet user's goals and to 
simulate a model whiteout bias and temporal deadlock^. Its main drawback is 
the strict hierarchical organization in levels. 

Holonic multi-agent systems (HMAS) can be viewed as a specific case of 
multi-level multi-agent-systems (MAS), the most obvious aspect being the loosely 

10 Practical aspects of this approach are discussed in |17| . E.g., AGVs embody the 
deadlock solving algorithm, becoming multi-level agents. This problem has been an 
important motivation in the development of IRM4MLS. 

11 Simulation properties of IRM4MLS may be exploited to explore model behavior us- 
ing, e.g., the polyagent concept [22121] . Such an approach may be used to determine 
fail probabilities of system components or control strategies. 



hierarchical organization of levels. However, from a methodological perspective, 
differences remain: thus, most of holonic meta-models focus on organizational 
aspects (cf. e.g., [2 6 35 34J). An important issue towards a generic meta-model 
for HMAS would be to define a holon with respect to IRM4MLS concepts: a 
holon cannot be defined with IRM4MLS first class abstractions (level, agent or 
environment), as it represents a multi- level entity. This situation is the main 
perspective of this work. 
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